Sample Bounded Distributed Reinforcement Learning for Decentralized POMDPs

Decentralized partially observable Markov decision processes (Dec-POMDPs) offer a powerful modeling technique for realistic multi-agent coordination problems under uncertainty. Prevalent solution techniques are centralized and assume prior knowledge of the model. We propose distributed reinforcement learning approach, where agents take turns to learn best responses each other’s policies. This promotes decentralization policy computation problem, relaxes reliance on full problem parameters. derive relation between sample complexity response error tolerance. Our key contribution is show that could grow exponentially with horizon. empirically even if requirement set lower than what theory demands, our approach can produce (near) optimal policies in some benchmark Dec-POMDP problems.